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Abstract 


How  does  the  underlying  data  affect  the  ability  of  warfighters  to  derive  useful 
information  and  make  decisions?  The  Army  Topographic  Engineering  Center  (TEC) 
and  GMU  endeavor  to  shed  light  on  this  question  with  the  third  in  TEC’s  series  of 
value  experiments.  The  fundamental  objective  of  the  series  is  to  improve  TEC’s 
support  of  military  personnel  in  the  field  through  better  geospatial  products.  The  third 
experiment  in  the  series  goes  in  a  different  direction  from  the  previous  two 
experiments,  which  were  presented  at  the  12th  and  13th  ICCRTS.  Whereas  previous 
experiments  assessed  the  value  of  cutting-edge  geospatial  tools  while  keeping  the  data 
constant,  the  present  experiment  evaluated  the  effect  of  higher  resolution  imagery  and 
elevation  data  while  keeping  the  tools  constant.  The  high  resolution  data  under 
evaluation  was  generated  from  TEC’s  Buckeye  system,  an  operational  airborne 
surveillance  system.  This  paper  discusses  the  scope  of  the  third  experiment,  its 
hypotheses,  its  experimental  design,  and  initial  results. 


As  researchers  and  developers  provide  increasingly  advanced  tools  to  process  data 
more  quickly  and  accurately,  it  is  necessary  to  assess  each  innovation  so  that  key 
resources  can  be  allocated  to  areas  that  yield  the  most  “bang  for  the  buck.”  To  meet  this 
need,  the  U.S.  Army  Engineer  Research  and  Development  Center  (ERDC)  initiated  the 
Joint  Geospatial  Enterprise  Services  (JGES)  program.  The  objective  of  JGES  is  to 
evaluate  the  value-added  to  military  decision-making  through  the  use  of  Geospatial 
Decision  Support  Products  (GDSPs).  GDSPs  are  computer-based  tools  that  allow  users 
to  access,  display,  and  reason  with  geospatial  data.  GDSPs  have  the  potential  to  provide 
superior  situation  awareness  through  the  use  of  tools  that  can  open  up  new  possibilities 
for  the  conduct  of  military  operations.  Implementing  Geospatial  technology  to  most 
effectively  support  the  warfighter  requires  a  spiral-build-test-build  development  cycle 
that  focuses  technological  efforts  in  directions  that  provide  the  most  value  to  the 
warfighter.  This  paper  reports  on  the  third  in  a  series  of  experiments  designed  to  assess 
the  value  of  geospatial  information  to  the  warfighter.  The  experimental  results 
concerning  high-resolution  data  from  the  Buckeye  system  will  be  use  to  guide  its  further 
development  and  ultimately  to  support  command  decisions  most  effectively. 


1.  Background 

Overhead  imagery  obtained  from  aircraft  was  used  extensively  for  military 
purposes  in  World  Wars  I  and  II.  Reconnaissance  aircraft  remained  the  primary  source 
for  overhead  imagery  until  the  advent  of  low  earth  orbit  (LEO)  surveillance  satellites.  As 
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early  as  1963,  satellites  generated  non-digital  imagery  with  a  9  -  25  foot  resolution,  and 
by  1967  the  resolution  had  improved  to  6  feet,  or  approximately  2  meters.  Although  this 
resolution  was  significantly  less  than  the  2.5  foot  (<  1  meter)  resolution  available  from 
the  U-2  reconnaissance  aircraft,  the  area  covered  by  the  imagery  from  a  single  satellite 
pass  dwarfed  the  total  imagery  collected  by  surveillance  aircraft.  Consequently,  satellite 
imagery  became  the  standard  source  for  reconnaissance  imagery  (Richelson  2003).  The 
development  of  digital  image  technology  provided  the  military,  and  later  the  public,  with 
access  to  digital  imagery  in  the  1 -meter  resolution  range.  The  military  standard  1 -meter 
resolution  imagery  is  designated  Controlled  Image  Base  1  (CIB1).  Google  Earth™  is  an 
example  of  1 -meter  resolution  digital  imagery  available  to  the  public.  While  1 -meter 
resolution  imagery  was  adequate  for  surveillance  and  military  planning  for  conflicts 
involving  large  units,  the  situation  is  different  for  asymmetric  warfare.  Asymmetric 
warfare  involves  small  groups  of  combatants,  and  requires  the  ability  to  recognize  objects 
with  dimensions  of  less  than  one  meter.  The  Buckeye  system  was  initially  developed  to 
provide  higher  resolution  imagery  (<  1  meter)  than  was  currently  available  to  facilitate 
the  asymmetric  battle  through  the  automated  change  detection  in  digital  imagery  (TEC 
2005). 


Until  relatively  recently,  elevation  data  has  historically  been  generated  by  manual 
survey.  The  completion  of  the  space  shuttle  radar  survey  generated  Digital  Terrain 
Elevation  Data  level  2  (DTED2)  for  the  Earth  between  60N  latitude  to  57S  latitude 
(Rabus  et  al.  2003).  DTED2  data  has  an  accuracy  of  +/-  30  meters  with  data  points  every 
30  meters  (Pike  2008).  DTED2  data  is  “bare  earth”  data;  it  has  been  processed  to 
eliminate  elevation  data  due  to  man-made  structures  (buildings,  bridges,  other  structures) 
and  flora  (trees  and  ground  cover).  DTED2  data  is  what  is  included  in  most  paper  and 
digital  topographic  maps. 

The  Buckeye  system  consists  of  two  components:  a  high-resolution  digital  camera 
to  generate  imagery,  and  a  Light  Detection  and  Ranging  (LIDAR)  system  to  generate 
elevation  data.  Buckeye  can  be  mounted  on  a  helicopter  or  an  Unmanned  Aerial  Vehicle 
(UAV).  Buckeye  provides  digital  color  imagery  with  a  4  to  6-inch  resolution  that  is 
orthorectified  (synchronized  with  known  geospatial  reference  systems).  This  imagery  is 
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of  higher  resolution  than  other  previously  available,  unclassified  aerial  reconnaissance 
imagery.  The  elevation  data  generated  by  Buckeye  is  comparable  to  Digital  Terrain 
Elevation  Data  level  5  (DTED5)  with  an  accuracy  of  +/-  1  meter  @  1  meter  spacing. 
Buckeye  LIDAR  data  is  not  “bare  earth.”  It  accurately  depicts  the  elevation  data 
associated  with  man-made  structures.  Buckeye  data  has  been  collected  for  most  of  the 
urban  areas  and  major  transport  arteries  in  Iraq.  Buckeye  data  is  unclassified  due  to  its 
unclassified  source,  but  is  treated  as  For  Official  Use  Only  (FOUO)  and  is  available  on 
the  military  SIPRNET  and  NIPRNET  (with  PKI)  from  TEC. 


As  a  vivid  depiction  of  the  combat  utility  of  Buckeye  imagery,  consider  the 
images  shown  in  Figures  1  and  2.  Figure  1  shows  CIB1  imagery  of  a  complex  whose 
purpose  cannot  be 


determined  from  the  image. 
A  Buckeye  image  of  the 
same  complex  is  shown  in 
Figure  2.  The  image 
clearly  shows  sports 
facilities  and  bleachers, 
strongly  suggesting  that  the 
compound  is  not  a  military 
target. 


Figure  1:  CIB1  Image  of  Complex 


The  goal  of  the  current  experiment  is  to  assess  the  benefits  of  Buckeye/FIDAR 
data  to  the  warfighter.  Specifically,  we  seek  to  assess  the  effects  of  higher  resolution  data 
on  military  decision-making.  We  investigated  two  aspects  of  how  experienced  military 
personnel  learn  from  imagery  and  elevation  data:  (1)  the  derivation  of  information  from 
data  and  (2)  the  evaluation  of  the  data  with  respect  to  a  specific  mission.  Both  these 
aspects  are  tasks  that  military  planners  routinely  undertake  when  evaluating 
imagery/elevation  data  and  both  aspects  are  well-defined  cognitive  processes  involved  in 
decision-making  theory.  In  his  revised  hierarchy  of  cognitive  processes,  Bloom  describes 
the  derivation  of  information  from  the  available  data  as  the  fourth  level  -  analysis;  and  the 
evaluation  of  information  and  data  as  the  fifth  of  his  six  levels  (Anderson  et  al.  2001). 
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imagery  and  elevation 
data  in  a  mission  specific 
context  requires  a  series 
of  decisions  and 
judgments  based  on 
written  policy  and 


experiment  captured  the 
impact  of  higher 
resolution  data  on 


experience. 


The  evaluation  of 


This 


Figure  2:  Buckeye  Image  of  a  School 


decision-making  in  a  military  planning  context.  This  was  achieved  by  quantifying  and 
measuring  the  participants’  ability  to  derive  information  from  and  evaluate  data. 


The  paper  is  organized  as  follows.  Section  2  describes  the  overall  scope 


of  this  experiment.  Section  3  discusses  the  primary  and  secondary  hypotheses  to  be 
examined.  Section  4  lays  out  the  design  of  the  experiment  and  the  reasoning  that  led  to 
this  design.  Section  5  discusses  the  computing  environment  to  be  used  in  the  experiment. 
Section  6  describes  the  metrics  to  be  used  to  quantify  the  results  of  each  trial.  Section  7 
discusses  the  results  and  the  impact  of  the  experiment. 

2.  Scope  of  Experiment 

Our  ultimate  objective  is  to  evaluate  the  benefit,  to  the  warfighter,  of  integrating 
higher  resolution  imagery  and  elevation  data  with  currently  available  Command  and 
Control  planning  tools.  This  third  experiment  sponsored  by  the  U.S.  Anny  Topographic 
Engineering  Center  (TEC)  for  the  Joint  Geospatial  Enterprise  Services  (J-GES)  program 
takes  a  different  approach  from  the  first  two  experiments,  presented  at  12th,  13th  and  this 
year’s  ICCRTS  conference  (Laskey  et  al.  2007;  Powell  et  al.  2008;  Powell  et  al.  2009). 
The  previous  two  experiments  evaluated  the  benefits  of  Geospatial  Decision  Support 
Systems  (GDSSs).  GDSS  are  a  subset  of  Geospatial  Decision  Support  Products  (GDSPs) 
that  perform  automated  analyses  of  geospatial  data  and  generate  geospatial  information, 
in  addition  to  displaying  data  and  information.  Those  experiments  varied  the  tool  set 
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while  keeping  the  resolution  of  the  data  constant.  In  contrast,  the  current  experiment 
evaluates  the  impact  of  higher  resolution  data  while  keeping  the  tool  set  constant. 

Discussions  with  Subject  Matter  Experts  (SMEs)  indicated  that  planners  for 
Battalion  sized  units  or  larger  (the  unit  size  of  previous  experiments)  were  unlikely  to 
benefit  from  Buckeye/LIDAR  data.  Planners  for  large  units  are  interested  in  large, 
operationally  significant  features  such  as  forests,  roads,  urban  areas,  and  rivers  and 
bridges.  Planners  for  small  units  are  interested  in  finer-grained,  tactically  significant 
features  such  as  trees  and  shrubs,  alleys  and  paths,  building  heights  and  walls,  and 
streams  and  fords.  Therefore,  small  unit  planners  would  probably  benefit  more  from  the 
higher  resolution  of  Buckeye/LIDAR  data  than  would  large  unit  planners.  In  this 
experiment,  the  general  scenario  asked  experienced  military  operators,  working 
individually  and  acting  as  small  unit  planners,  to  evaluate  multiple  potential  sites  for  a 
Vehicle  Control  Point  (VCP).  Follow-on  experiments  will  address  additional  kinds  of 
planning  problems,  at  various  levels  of  command,  and  involving  collaboration  among 
members  of  staffs  as  well  as  individual  decision  makers. 

3,  Hypotheses 

As  we  discovered  while  planning  the  first  experiment,  in  order  to  evaluate  the 
military  value  of  GDSPs  we  needed  a  clear  definition  of  military  value,  together  with 
quantifiable  metrics  of  value.  Our  determination  of  what  constitutes  value  in  this 
experiment  is  based  on  discussions  with  several  experienced  military  SMEs.  These 
planners  believe  that  the  value  of  GDSPs  lie  in  their  ability  to: 

(1)  Reduce  the  time  spent  evaluating  an  area.  Because  the  higher  resolution  of 
Buckeye/LIDAR  data  should  reduce  uncertainty  about  the  terrain,  participants 
should  be  able  to  spend  less  time  subjectively  estimating  the  impact  of  the 
uncertainty  on  the  mission  than  when  using  CIB1/DTED2.  Less  uncertain 
data  should  also  allow  the  participants  to  fonn  their  overall  evaluations  more 
rapidly. 

(2)  Improve  the  operator ’s  ability  to  extract  meaningful  information.  As  the 
resolution  of  imagery  and  elevation  data  improves,  the  data  becomes  more 
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(3)  Improve  the  operator ’s  ability  to  evaluate  each  site.  As  the  higher  resolution 
Buckeye/LIDAR  data  has  less  uncertainty,  evaluators  using  it  should  be  able 
to  better  evaluate  a  site’s  value  as  a  VCP  site.  The  Buckeye  imagery  should 
provide  evaluators  with  better  visual  data  on  the  structures  and  the  condition 
of  the  structures  present  at  the  site.  The  LIDAR  data  should  provide  the 
evaluators  with  better  information  on  the  topography  of  the  site  including  the 
heights  of  buildings  and  obstacles  that  would  obstruct  fields  of  view. 

(4 )  Increase  the  uniformity  of  participants’  responses.  Since  the  participants’ 
judgments  are  based  on  higher-resolution,  less  uncertain  information,  their 
assessment  of  the  factors  contributing  to  the  quality  of  potential  VCP  sites 
should  be  more  accurate.  This  is  expected  to  reduce  the  variability  in  their 
responses,  because  the  responses  should  cluster  around  an  accurate  assessment 
of  the  value  of  the  site. 

It  follows  from  the  criteria  above  that,  in  comparison  with  decision-makers  using 
higher  resolution  data,  we  hypothesize  that  trained,  experienced,  military  planners  who 
use  Buckeye/LIDAR  data,  in  comparison  with  those  using  CIB 1/DTED2,  would: 

HI.  Evaluate  the  data  more  quickly.  Rationale:  Higher  resolution  data  reduces 
the  uncertainty  associated  with  the  information  upon  which  the 
participant’s  evaluations  are  based  and  thus  their  evaluations  should 
require  less  time. 

H2.  Require  less  additional  information  to  establish  a  VCP  .  Rationale:  The 
higher  resolution  data  should  provide  more  information  for  the 
participant’s  evaluation  and  thus  they  should  assess  that  less  additional 
information  is  required  from  external  sources. 
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H3.  Be  able  to  more  accurately  derive  information.  Rationale:  When  using  the 
higher  resolution  Buckeye/LIDAR  elevation  data,  the  participants  should 
be  better  able  to  use  the  available  GDSS  to  answer  questions  about  the 
sites  more  accurately. 

H4.  Be  more  uniform  in  their  evaluations,  i.e.  have  less  variance,  in  two  of  the 
four  categories  above  (better  information  and  better  evaluation). 
Rationale:  Using  higher  quality  information  derived  from  less  uncertain 
data  should  cause  the  participants  evaluations  (each  criterion  and  overall) 
to  agree  more  closely. 

As  the  determination  of  military  value  and  the  design  of  the  experiment  evolved, 
we  identified  two  secondary  hypotheses.  First,  the  structure  of  the  experiments  requires 
the  repetition  of  evaluations  and  there  was  concern  that  a  learning  effect  might  skew  the 
results  of  the  experiment.  Second,  although  not  a  concrete  benefit,  the  perception  of  the 
participants  as  to  the  specific  utility  of  the  Buckeye/LIDAR  data  will  assist  in  the 
integration  of  Buckeye  data  into  deployable  systems.  The  secondary  hypotheses 
investigated  include: 

H5.  There  would  not  be  a  learning  effect  due  to  experimental  design. 
Rationale:  The  participants  have  previous  training  and  experience  using 
C2  planning  tools  and  the  tasks  the  participants  are  asked  to  perform  are 
similar  to  those  that  they  have  performed  in  the  normal  course  of  their 
duties. 

H6.  Participants  would  consider  using  Buckeye  superior  with  respect  to  speed, 
ease  of  use,  usefulness  of  information,  and  overall.  Rationale:  The 
participants  should  consider  the  high-resolution  Buckeye/LIDAR  data  of 
benefit  in  the  planning  process. 

4.  Study  Design 

The  experiment  employed  a  factorial  design  with  three  independent  variables: 
Data  Source  (Buckeye  or  CIB1),  Data  Order  (whether  the  first  scenario  is  worked  with 


8  of  20 


Buckeye  data  or  CIB 1  data),  and  Scenario  order  (whether  scenario  1  or  2  is  worked  first). 
Data  Source  was  a  within-subject  variable  because  each  participant  worked  one 
independent  planning  scenario  with  CIB1/DTED2  and  one  independent  scenario  with 
Buckeye/LIDAR  data.  A  within-subject  design  is  particularly  valuable  when  the  number 
of  available  participants  is  limited,  as  in  the  current  case.  Results  from  the  sets  of  tasks 
can  be  compared  for  each  participant,  thus  eliminating  participant-specific  effects  that 
might  add  variability  to  the  results.  Data  Order  and  Scenario  order  were  between- 
subjects  variables  because  any  given  participant  can  only  experience  one  ordered 
sequence  for  these  variables  without  repeated  exposure  to  both  data  sources. 

The  participants  evaluated  sets  of  three  potential  VCP  sites.  A  VCP  is  a 
checkpoint  on  a  road  where  vehicles  are  stopped  and  searched.  The  participants 
evaluated  all  sites  using  the  same  underlying  C2  system,  the  Commander’s  Support 
Environment  (discussed  in  section  5).  One  set  of  three  sites  was  evaluated  using 
Buckeye/LIDAR  data,  and  a  second  set  of  three  sites  was  evaluated  using  CIB1/DTED2 
data.  A  third  evaluation  re-evaluated  the  sites  originally  evaluated  with  CIB1/DTED2, 
this  time  using  Buckeye/LIDAR  data.  This  third  evaluation  provided  a  vehicle  for 
directly  comparing  the  participants’  evaluations  on  the  same  site,  but  with  different 
imagery  and  elevation  data.  Because  the  judgments  may  have  been  biased  by  having 
seen  the  sites  previously,  this  direct  comparison  was  not  our  primary  comparison. 
Nevertheless,  the  direct  comparison  provides  information  about  whether  participants’ 
evaluations  of  the  sites  improved  when  CIB1/DTED2  data  was  replaced  by 
Buckeye/LIDAR  data.  All  the  trials  are  essentially  identical  except  for  the  source  of  the 
imagery  and  elevation  data.  The  Data  Source  for  the  evaluations  was  randomly  selected 
so  that  half  of  the  participants  used  Buckeye  and  LIDAR  first.  Randomizing  the  order  of 
the  tasks  enabled  the  analysis  to  control  for  and  evaluate  learning  effects. 

The  instructions,  sites,  evaluation  criteria,  and  tools  were  the  same  in  both 
scenarios  with  the  exception  of  geographic  references  necessitated  by  the  requirement  to 
have  different  geographic  areas  for  each  trial.  Different  geographic  areas  are  required  to 
prevent  participants  from  repeating  their  responses  from  the  first  scenario  when  they  form 
responses  for  the  second  scenario.  Having  the  participants  evaluate  three  sites  in  each 
trial  was  advantageous  in  two  ways:  (1)  we  were  able  to  analyze  the  participants’ 
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evaluations  by  individual  site  and  directly  compare  their  evaluations;  and  (2)  by 
averaging  each  participant’s  responses,  the  impact  of  variations  in  each  site  could  be 
minimized.  The  trios  of  sites  have  been  carefully  selected  for  their  geographic  similarity 
such  that  the  evaluations  performed  by  the  participants  and  the  expected  results  were  as 
nearly  identical  as  possible.  Randomization  was  used  to  control  for  differences  between 
scenarios. 

The  participants  were  Army  enlisted  personnel  and  officers  who  have  previous 
experience  establishing  VCPs  in  Iraq  or  Afghanistan.  They  were  split  into  two  groups 
that  were  as  evenly  balanced  with  respect  to  ability  rank/time  in  service  as  possible.  Of 
the  fifteen  U.S.  Army  subjects,  eleven  were  stationed  at  Ft.  Lewis,  WA,  and  four  were 
stationed  at  Ft.  Benning,  GA.  Five  participants  were  majors  and  ten  were  enlisted  (six 
Staff  Sergeants,  three  sergeants,  one  specialist).  Fourteen  were  active  duty  and  one  was 
retired.  Further  evaluation  of  the  relative  ability/experience  of  the  participants  was  not 
possible  due  to  the  inability  to  contact  the  participants  prior  to  conducting  the  trials. 
Group  I  performed  the  evaluations  first  with  Buckeye/  LIDAR  and  then  with 
CIB1/DTED2.  Group  II  performed  the  evaluations  in  the  reverse  order.  The  group  was 
further  divided  into  two  subgroups  while  maintaining  the  balance  of  ability  and 
knowledge.  Each  subgroup  performed  the  same  evaluations  for  the  same  two  scenarios, 
but  the  two  subgroups  saw  the  two  scenarios  in  the  opposite  order.  This  design  allowed 
us  to  control  for  differences  due  to  the  order  of  system  use  and  the  scenario  order. 

Each  trial  consisted  of  evaluating  one  of  two  sets  of  three  similar  potential  VCP 
sites  (a  scenario)  on  the  same  criteria.  There  were  28  evaluation  criteria  divided  into  six 
categories.  The  questions  were  derived  from  a  U.S.  Marine  Corps  battalion  Standard 
Operating  Procedure  (SOP).  In  each  evaluation,  the  participants  evaluated  each  site  on 
the  28  criteria  with  respect  to  the  amount  of  additional  information  that  would  be  required 
to  actually  establish  a  VCP  at  each  site.  For  each  site,  the  participants  also  answered  four 
questions  that  only  required  that  they  derive  information  using  the  tools  inherent  in  CSE. 
After  completing  the  evaluation  of  all  three  sites  in  a  scenario,  the  participants  ranked  the 
sites  relative  to  one  another  on  the  overall  quality  of  the  site  for  a  VCP  and  estimated 
their  confidence  in  their  ranking. 
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After  completing  all  three  trials,  the  participants  weighted  the  relative  importance 
of  the  categories  and  criteria.  Because  participants  may  weigh  the  various  criteria 
differently,  these  rankings  may  help  in  exploring  which  criteria  are  most  important  in 
their  evaluations  and  help  us  to  compare  the  differences  in  the  rankings  of  each  site. 
Finally,  the  participants  completed  a  questionnaire  comparing  the  relative  benefits  of 
Buckeye/LIDAR  and  CIB1/DTED2  in  the  areas  of  speed,  ease  of  use,  utility  of  the 
information,  and  overall. 

Prior  to  beginning  the  tasks,  both  groups  of  participants  received  standardized 
training  on  the  use  of  CSE.  The  training  was  sufficient  to  allow  the  participants  to 
perform  the  required  evaluations  and  included  training  on  the  tools  and  features  unique  to 
CSE.  The  last  phase  of  the  training  required  the  participants  to  perform  the  complete 
evaluations  of  two  training  sites  similar  to  those  that  the  participants  encountered  during 
the  trials. 

5.  Environment 

The  computers  used  for  the  experiments  were  not  homogeneous:  four  Dell 
desktops,  four  Dell  XPS  laptops,  two  Prostar  laptops,  and  two  other  Dell  laptops  were 
available.  The  laptops  were  configured  such  that  the  monitor  resolution  and  area 
displayed  were  near  to,  but  not  less  than,  that  of  the  desktops.  All  the  computers  were 
dual  core  with  greater  than  2.0  GHz  processors  and  USB  2.0  capability.  Because 
input/output  operations  (I/O)  are  approximately  1000  times  slower  than  accessing  data 
from  RAM,  the  limiting  factor  in  the  display  of  the  imagery  and  elevation  data  was  the 
time  required  to  access  the  data  from  the  external  hard  drive.  Ensuring  that  all  the 
computers  were  using  USB  2.0  minimized  variation  in  the  response  time  due  using 
heterogeneous  computers.  The  laptops  were  provided  with  mice  so  that  participants 
would  not  be  required  to  use  touch  pads.  To  control  for  any  remaining  variation  due  to 
the  participants  using  specific  computers,  the  participants  used  a  randomly  assigned 
computer  for  each  trail. 

The  evaluation  was  conducted  using  the  Commanders  Support  Environment 
(CSE)  as  the  Command  and  Control  (C2)  planning  system.  CSE  is  a  robust  C2  planning 
and  execution  system  developed  for  experimentation.  The  CSE  was  originally  developed 
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for  Defense  Advanced  Research  Projects  Agency  (D  ARP  A)/ Army  Multi-Cell  and 
Dismount  C2  Program  (M&D  C2).  M&D  C2  is  a  continuation  of  the  Future  Combat 
System  Command  and  Control  (FCS  C2)  program,  and  hosted  a  series  of  experiments 
designed  to  test  out  network  centric  warfare  concepts.  The  CSE  is  primarily  written  in 
C++  code  for  the  Microsoft  Windows  environment.  It  is  built  upon  the  Viecore  FSD 
Decision  Support  System  (VDSS),  and  the  Data  Analysis  and  Visualization  Infrastructure 
for  C4i  (Davinci)  Toolkit.  The  VDSS  architecture  enables  the  quick  addition  of  modules 
for  communication  between  CSE  and  other  systems  and  components.  The  CSE’s  GIS 
components  are  built  upon  the  Commercial  Joint  Mapping  Toolkit(C/JMTK)  which 
includes  ESRI’s  ArcGIS  Desktop  licensed  at  the  ArcEditor  level. 

In  addition  to  its  ability  to  display  imagery  and  elevation  data,  the  CSE  provides 
one  primary  GDSS,  an  optimized  Line  of  Sight  (LOS)  analysis  tool.  The  LOS  tool 
performs  real-time  line-of-sight  analysis  based  on  the  relevant  digital  elevation  data,  and 
displays  the  results  as  either  a  360°  fan  or  an  elevation  cross  section  out  to  5  km  from  the 
cursor. 

6.  Metrics 

The  criteria  for  evaluation  were:  (1)  the  speed  with  which  evaluations  were 
conducted;  (2)  the  need  for  additional  information;  (3)  the  accuracy  of  the  derived 
information;  and  (4)  the  perception  of  the  participants  regarding  the  relative  merits  of  the 
Buckeye/LIDAR  and  CIB1/DTED2  data.  The  participants  either  recorded  their 
evaluations  of  each  of  the  28  criteria  on  a  5-point  Likert  scale  or  provided  short  answers 
to  questions.  To  ensure  anonymity,  participants  were  assigned  participant  numbers  and 
evaluation  designators,  and  data  were  recorded  by  these  designators. 

Time  to  Completion  (HI,  H4,  H5).  The  evaluation  of  how  quickly  the 
participants  completed  their  evaluations  was  measured  objectively  by  logging  the  amount 
of  time  it  takes  participants  to  complete  the  tasks.  The  maximum  duration  of  each  trial 
was  1.5  hours.  The  actual  time  was  calculated  by  taking  the  difference  between  the  start 
and  stop  times  and  subtracting  any  break  time. 

Additional  Information  (H2,  H4,  H5,).  In  military  parlance,  a  commander 
would  issue  a  Request  For  Information  (RFI)  to  higher  authority  for  this  information. 
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Depending  on  the  amount  of  information,  the  detail  requested  and  the  source  required,  a 
response  to  an  RFI  can  consume  man-hours  of  effort  by  numerous  people  in  multiple 
external  agencies,  and  thus  may  significantly  delay  the  mission.  The  participants 
evaluated  each  site  on  the  28  criteria  on  a  5-point  Likert  scale  where  a  1  is  “significant 
additional  information  required”  and  5  is  “no  additional  infonnation  needed.”  This 
metric  is  a  subjective  judgment  based  on  the  participants’  analysis  of  the  amount  of 
information  contained  in  the  data. 

Accuracy  of  derived  information  (H3).  We  asked  the  participants  to  answer 
four  questions  about  each  site  to  determine  how  higher  resolution  data  affected  the 
participants  ability  to  derive  accurate  information  from  the  data.  Two  of  the  questions 
require  that  the  information  be  derived  from  an  examination  of  the  imagery  and  two 
require  that  the  participants  use  the  LOS  GDSP  (acting  on  the  elevation  data)  to  derive 
the  information.  Unlike  the  RFI  evaluation,  these  questions  are  objective  in  that  there  is  a 
right  answer  and  little  or  no  subjective  analysis  is  required  on  the  part  of  the  subjects. 
Deriving  this  information  is  typical  of  the  many  individual  tasks  that  are  required  to  make 
the  overall  evaluations.  The  information  generated  by  the  subjects  was  compared  to 
“ground  truth”  answers  derived  using  all  the  available  data. 

Perception  of  Merits  (H6).  We  administered  a  questionnaire  to  evaluate  the 
participants’  subjective  judgment  of  the  benefits  of  Buckeye/LIDAR  data  as  compared  to 
CIB1/DTED2  data.  The  participants  evaluated  which  data  is  more  beneficial  as  to  speed, 
ease,  and  value  of  information  with  respect  to  the  imagery  and  the  elevation  for  nine  tasks 
and  overall.  Like  the  evaluation  criteria,  the  participants  evaluated  the  participants’ 
answers  on  a  5 -point  Likert  scale.  The  results  of  these  questions  and  infonnation 
gathered  in  a  debriefing  session  conducted  at  the  conclusion  of  the  experiment  will  be 
particularly  valuable  in  guiding  the  future  integration  of  Buckeye/LIDAR  data  with 
deployed  systems. 

Area  Characteristic  Ratings,  One  additional  metric  that  had  potential  to  help 
understand  the  benefits  of  high-resolution  data  was  an  Area  Characteristic  score.  The 
participants  would  rate  each  site  on  the  same  28  criteria  as  they  did  the  RFI  metric,  but 
instead  rate  how  “good”  the  site  was  with  respect  to  the  criteria  and  mission.  In  order  to 
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control  for  the  difference  in  terrain  at  each  site,  the  participants’  scores  would  have  to  be 
compared  to  a  “ground  truth”  score  for  each  criterion  for  each  site.  There  is  no 
objectively  correct  ’’ground  truth”  for  these  ratings,  and  therefore  there  is  no  way  to 
evaluate  objectively  whether  Buckeye  enabled  participants  to  produce  better  area 
characteristic  ratings.  As  a  surrogate  for  “ground  truth,”  three  SMEs  were  tasked  with 
evaluating  each  site  individually  and  arriving  at  consensus  scores. 

This  approach  proved  infeasible  due  to  the  lack  of  agreement  on  the  part  of  the 
SMEs  as  to  what  constituted  “good”  for  each  criterion.  The  SMEs  consisted  of  a 
Captain,  a  retired  Staff  Sergeant,  and  a  Corporal.  Their  experience  with  VCPs  ranged 
from  Commander,  platoon  sergeant  to  fire  team  leader.  The  attitude  in  Iraq  during  their 
deployments  ranged  from  attempting  to  incite  a  response  from  insurgents  to  actively 
avoiding  disrupting  the  day-to-day  life  of  the  populace.  Their  wide  range  of  experiences 
contributed  to  widely  varying  judgments  as  to  what  constituted  a  good  VCP.  The 
average  correlation  among  the  SMEs  was  0.4.  This  low  correlation  diminishes 
confidence  in  their  consensus  score.  As  a  consequence,  this  metric  is  not  considered  in 
the  analyses  reported  below. 

7.  Analyses  of  Results. 

Hypothesis  1:  Time  to  Completion.  The  statistical  analysis  of  the  time  to 
completion  yielded  important  insights.  A  repeated-measures  ANOVA  indicated  strong 
statistical  evidence  (p  =  0.01)  that  the  average  time  to  completion  differs  for  the  two  data 
resolutions  (Table  1).  Contrary  to  our  hypothesis  1,  on  average,  participants  took  more 
time  to  complete  the  evaluations  when  using 
Buckeye/LIDAR  than  when  using 
CIB1/DTED2.  Discussions  with  the 
participants  after  completion  of  the  trials 
indicated  that  the  higher-resolution  data  of 
Buckeye/LIDAR  allowed  a  more  detailed  analysis  of  each  site.  Consequently,  the 
evaluations  with  this  higher  resolution  data  took  additional  time.  Although  significant 
statistically,  the  overall  average  difference  was  only  four  minutes  for  each  trial,  less  than 
10%  of  the  total  time  per  trial. 


Time  to  Completion  (Minutes) 


X 

s 

CIB1/DTED2 

47.40 

6.080 

Buckeye/LIDAR 

51.67 

9.499 

Table  1:  Time  to  Completion  Data 
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Hypothesis  2:  Additional  Information  (RFI).  A  repeated  measure  ANOVA 
conducted  on  the  averages  of  the  81  data  points  for  each  participant  for  each  trial  (27 
criteria  x  3  sites)  resulted  in  strong  evidence  (p  <  0.001)  that,  on  average,  participants 
required  less  additional  infonnation  when  using  Buckeye/LIDAR  data  than  when  using 
CIB1/DTED2  data.  The  RFI  metric  is  a  proxy  for  the  value  of  the  information  contained 
in  the  data.  The  relationship  between  RFI  data  and  the  value  of  information  is  an  inverse 
one;  that  the  participants  required  less  additional  information  with  Buckeye/LIDAR  data 
implies  that  the  participants  acquired  more  information  from  this  data  than  from 
CIB1/DTED2  data.  As  the  data  for  both  Buckeye/LIDAR  and  CIB1/DTED2  covers  the 
same  geographic  area  and  features,  the  finer-grained  Buckeye/LIDAR  data  provided 
more  information  and  was  consequently  more  valuable  than  the  CIB1/DTED2  data. 

Hypothesis  3:  Derived  formation.  In  support  of  our  hypothesis,  analyses 
indicated  that  participants  were  able  to  derive  infonnation  more  accurately  using 
Buckeye/LIDAR.  Table  2  shows  the  percentage  of  correct  responses  over  all  four 
questions  (Overall),  for  overall  Elevation  and  Imagery  data,  and  by  question.  Pearson 
Chi-Squared  tests  were  conducted  that  compared  the  number  of  correct  and  incorrect 
answers  to  the  questions  that  were 
designed  to  tests  the  accuracy  of 
information  derived  from  the  digital 
data.  There  is  strong  statistical 
evidence  (p  <  0.001)  that,  for  both 
imagery  and  elevation  data  from  all 
sites,  participants  were  able  to  more 
accurately  derive  information  from 
the  Buckeye/LIDAR  data  than  from 
CIB1/DTED2  data.  Overall,  the 
participants  using  Buckeye/LIDAR  data  generated  correct  responses  approximately  73% 
of  the  time,  as  compared  to  just  under  16%  when  using  CIB1/DTED2.  This  was  as 
expected,  as  the  empirical  evidence  from  post  trial  debriefs  with  the  participants  indicated 
enthusiasm  for  the  Buckeye  imagery  and  LIDAR  data. 


Percentage  of  Correct  Responses 


Buckeye 

LIDAR 

CIB1 

DTED2 

Overall 

72.80% 

15.60% 

Elevation 

74.40% 

23.40% 

Ql 

62.20% 

13.40% 

Q2 

86.60% 

33.40% 

Imagery 

71.20% 

7.80% 

Q3 

75.60% 

11.20% 

Q4 

66.60% 

4.40% 

Table  2:  Derived  Information  Data 
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Hypothesis  4:  Uniformity. 

There  was  no  statistical  evidence  to  support  our  hypothesis  that  using  higher- 
resolution  data  would  result  in  less  variable  evaluations. 


Time  to  Completion:  Data  Order 
CIB1/DTED2 


Hypothesis  5:  Learning  Effect.  There  was  strong  statistical  evidence  (p  =  0.01) 
of  a  Data  and  Data  Order  interaction.  The  average  time  to  completion  when  using  both 
data  resolutions  was  longer  for  the  first  data 
set  used,  indicating  that  the  participants  were 
learning  about  the  problem  during  the  trials 
(Table  3).  This  effect  is  probably  due  to  the 
evaluation  tasks  being  similar  but  not 
identical  to  the  tasks  participants  performed 
when  actually  setting  up  VCPs.  In  the  field, 
the  final  evaluations  of  VCP  sites  are  typically 
done  on  site  and  the  evaluator  would  have 


X 

s 

CIB1/DTED2  First 

49.71 

6.237 

Buckeye/LIDAR  First 

45.38 

5.528 

Time  to  Completion:  Data  Order 
Buckeye/LIDAR 


X 

s 

Buckeye/LIDAR  First 

54.50 

10.184 

CIB1/DTED2  First 

48.43 

8.162 

Table  3:  Time  to  Completion  Interactions 
first  hand  information  in  addition  to  digital  geospatial  data.  Although  the  participants 

were  trained  on  sites  similar  to  those  evaluated,  and  pilot  testing  indicated  that  the 

training  was  sufficient,  these 


results  indicate  that  participants 
continued  to  learn  how  to 
evaluate  the  available  geospatial 
information  throughout  the 
experiment. 

Figure  3  summarizes  the 
analyses  we  conducted.  The  two 
points  for  the  Buckeye-CIBl 
Data  Order  (left  side  of  Figure 
3)  indicate  a  definite  difference 
between  the  average  times  for 
trials  when  the  data  order  was 
Buckeye/LIDAR  first  and 
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Figure  3:  Time  to  Conmletion  Interaction 
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CIB1/DTED2  second.  This  difference  in  the  average  times  is  due  to  the  compound  effect 
of  the  longer  time  required  when  using  Buckeye  and  during  a  first  trial,  as  compared  to 
the  shorter  time  required  both  when  using  CIB1/DTED2  and  during  a  second  trial. 
Conversely,  the  two  points  for  CIB1 -Buckeye  Data  Order  (right  side  of  Figure  3)  shows 
little  difference  for  the  two  trials.  The  longer  time  required  for  a  Buckeye/LIDAR  trail 
seems  to  have  been  offset  by  the  shorter  time  required  on  a  second  trail. 

Hypothesis  6:  Perception.  From  the  questionnaire  data,  there  is  evidence  that 
the  participants  of  the  experiment  considered  Buckeye/LIDAR  superior  to  CIB1/DTED2 
in  that  they  believed  it  allowed  them  to  compete  the  tasks  faster,  that  it  made  the  tasks 
easier,  that  the  information  was  more  useful,  and  that  it  was  superior  overall.  Figure  4 
graphically  depicts  this.  In  post  trial  debriefs  most  participants  indicated  that  they  were 
genuinely  impressed  with  the  level  of  detail  of  Buckeye/LIDAR  data. 


Comparison  of  Resolutions 
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Figure  4:  Conmarison  Results 


Conclusions.  This  experiment  provided  significant  insight  into  the  benefits  of 
high-resolution  imagery  and  elevation  data.  The  primary  benefits  noted  were: 

1 .  High-resolution  imagery  and  elevation  data  provided  more 
information.  In  all  cases,  the  participants  indicated  they  would  require 
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less  additional  information  if  they  were  to  actually  complete  the 
mission. 

2.  High-resolution  imagery  and  elevation  data  allowed  the  participants  to 
derive  more  accurate  information.  In  all  cases,  participants  using  the 
higher  resolution  data  answered  questions  about  sites  more  accurately. 

3.  The  participants  believe  that  using  high-resolution  data  allows  them  to 
complete  their  evaluations  more  quickly,  made  the  evaluations  easier, 
contained  more  useful  information,  and  was  superior  overall. 

Two  other  interesting  effects  were  noted: 

1.  High-resolution  imagery  and  elevation  data  slowed  the  evaluation 
process  by  about  9%.  This  effect  was  probably  due  to  the  time 
required  to  assess  the  increased  information  provided  by  this  data. 

2.  There  was  a  learning  effect.  Even  though  the  participants  had  previous 
experience  with  imagery  and  the  training  they  received  had  been 
assessed  as  sufficient,  the  participants  continued  to  learn  how  to 
evaluate  the  high-resolution  data  throughout  the  trials. 

The  results  of  this  experiment  are  encouraging.  Although  the  participants  took 
four  minutes  longer  to  analyze  the  high-resolution  Buckeye/LIDAR  data,  the  potential 
saving  in  reduced  RFIs  is  enormous.  Processing  RFIs  can  be  so  laborious  that  the  time 
and  resources  saved  in  the  overall  planning,  both  inside  and  external  to  the  unit,  would 
offset  the  slight  increase  in  time  spent  analyzing  high-resolution  data.  For  instance,  a 
Marine  sniper  who  participated  in  the  experiment  pilot  test  indicated  that  if  he  had  access 
to  Buckeye/FIDAR  data,  he  could  have  shaved  two  days  off  a  5 -day  reconnaissance 
mission. 

Additionally,  the  reduced  uncertainty  in  this  high-resolution  data  allowed  the 
participants  to  glean  more  accurate  information  from  the  data.  The  participants  were  able 
to  use  the  GDSPs  to  more  accurately  derive  useful  information  from  both  imagery  and 
elevation  data.  The  information  available  from  the  imagery  allowed  the  participants  to 
better  conceptualize  the  environment  in  which  the  VCPs  were  to  be  established.  In 
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response  to  specific  questions,  they  were  able  to  estimate  vehicular  and  pedestrian  traffic 
and  determine  how  the  urban  terrain  could  help  or  hinder  channeling  traffic.  Participants 
stated  that  the  more  accurate  elevation  data,  and  the  Line  of  Sight  (LOS)  information 
generated  from  it,  would  be  valuable  in  force  protection  decisions  such  as  the  placement 
of  overwatch  and  sniper  positions. 

Overall,  the  results  of  the  experiment  help  to  detennine  the  specific  benefits  to  the 
warfighter  of  high-resolution  imagery  and  elevation  data.  Many  of  the  benefits  can  be 
generalized  to  missions  other  than  the  mostly  defensive  mission  of  establishing  a  VCP. 
Participant  comments  indicated  that  they  saw  value  in  using  Buckeye/LIDAR  data  in 
other  small  unit  operations  such  as  routine  patrols,  assaults  on  fixed  positions, 
reconnaissance,  and  intelligence  gathering.  Further  analysis  of  the  individual  criteria  RFI 
data  will  likely  yield  specific  areas  where  Buckeye/LIDAR  data  would  be  useful  and 
where  future  development  efforts  and  experiments  can  be  concentrated. 
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Background 


•  Geospatial  is  focal  point  of  military 
planning 

•  Geospatial  Decision  Support  Products  are 
rapidly  penetrating  all  command  levels 


•  Empirical  research 
is  needed  to: 


-  Evaluate  military  value 
of  emerging  products 

-  Prioritize  future  product 
development 


Why  Conduct  Experiments? 


•  Most  military  R  &  D  tests  to  requirements 

•  Hypothesis  driven  to  test  value-added 

•  Statistically  significant  results 

-  Quantitative  not  just  qualitative  feedback 

•  Answer  questions: 

-  What  is  the  value  added  for  the  warfighter? 

-  Does  the  product  meet  operational  needs? 

-  How  can  the  product  be  improved? 


Purpose  of  Research  Program 


•  Sponsored  by 

-  U.S.  Army  Engineer  Research  and  Development  Center 
(ERDC) 

-  U.S.  Army  Geospatial  Center  (AGC) 

•  Purpose: 

-  Assess  the  value-added  to  Military  Decision  Making  from 
use  of  Geospatial  Decision  Support  Products  (GDSPs) 

-  Evaluate  the  value-added  of  the  Buckeye/LIDAR  high- 
resolution  imagery  and  elevation  data 


Buckeye  /  LIDAR 


Objective: 

Provide  unclassified  high-resolution  geospatial  data  that  can  be  applied 
to  tactical  missions 


Products  -  High  Resolution  Data 

-  Buckeye 

•  10-15  cm  (4-6  in)  resolution  color  digital  imagery 

-  LIDAR 

•  Digital  Terrain  Elevation  Data  level  5  (DTED5)  comparable  elevation 
data 

•  Elevation  data  +/-  1  meter  at  1  meter  spacing 

-  Co-located  on  helicopter  /  UAV 


Buckeye/LIDAR  products  are  currently  available  in  theater  on 
the  NIPR  and  SIPR  nets 


38,000  sq  km  data  on  Iraqi  urban  areas  and  supply  routes 
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What  is  it? 


^Without  Buckeye? 


Controlled  Image  Base  -  1  meter  (CIB1) 


UNIVERSITY 


Buckeye  Imagery 


4^With  Buckeye? 


Looks  like  a  school  ^ 


Current  Study 


•  Study  Objective 

-  Assess  the  benefits  of  Buckeye/LIDAR  to  military 
planners  in  a  complex  and  realistic  scenario 


-  To  determine  the  effect  of  high-resolution  data  on 
military  decision-making 


-  Different  approach  from  two  previous  experiments 
(presented  at  12th,  13  th,  14thICCRTS) 

•  Varied  the  resolution  of  data  while  maintaining  computer 
tools  constant. 
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Study  Method 


Mission: 

-  Evaluate  three  potential  sites  for  Vehicle  Control  Point 

-  Infantry  platoon 

-  Actual  in-country  urban  data 

-  Three  sites  per  trial 

Three  trials 

1) With  Buckeye/LIDAR  data 

2)  With  CIB1/DTED2  data 

3) CIB1/DTED2  trial  with  Buckeye/LIDAR  data 


Hypotheses 


1. 

2. 

3. 

4. 

5. 


Participants  who  use  the  Buckeye/LIDAR  would  be  able  to 
derive  information  more  accurately 

Participants  who  use  the  Buckeye/LIDAR  would  require  less 
additional  information  in  order  to  actually  establish  a  VCP 

Participants  who  use  the  Buckeye/LIDAR  would  produce  output 
more  quickly 

There  will  be  little  or  no  learning  effect  due  to  evaluation 
design 

Participants  will  consider  using  the  Buckeye/LIDAR  superior 
with  respect  to  speed,  ease  of  use,  usefulness  of  information  and 
overall 


1. 

2. 

3. 

4. 

5. 
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Study  Design 


•  Within  Participants  design  with  respect  to  System  used: 

Each  subject  will  evaluate  scenarios  consisting  of  three  sites  in  both 
conditions  (with  Buckeye/LIDAR  data  and  with  CIB1/DTED2  data) 

•  Between  Participants  design 

-  System  Order  (which  system  is  used  first) 

-  Scenario  Order  (which  scenario  is  used  first)_ 

Design  was  counterbalanced  on  scenario  order  and  system  order 

•  Study  design  will  maintain  the  required  statistical  power  and 
minimize  the  number  of  participants 

•  Training  prior  to  trials 

-  CSE  (1  hour)  and 

-  Buckeye/LIDAR  (1/2  hour) 

-  Sample  evaluations  (1  hour) 


Study  Design  (cont) 


-Participants 

-  15  U.S.  Army  Personnel 

•  In  country  experience  establishing  VCPs 

•  Experienced  varied:  command,  platoon  Sgt,  fire  team  leader 

•  Ft.  Lewis  (11)  and  Ft.  Benning  (4). 

-  Anonymous 

•  Randomly  assigned  participant  numbers 

•  Randomly  assigned  data  designators 

-  Experience  Questionnaire 

•  Unable  to  control  for  experience 

•  Post  Hoc  analysis 

-  Randomly  assigned  to  groups 
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Experimental  Tasks 
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Evaluate  three  sites  as  to  its  potential  for  establishing 
a  VCP 


Specific  tasks  : 

-  Evaluate  the  potential  of  each  site  on  28  criteria  in  6 
categories 

•  Area  Characteristics  (guidance) 

•  Requests  for  additional  information  (RFIs) 


Respond  to  questions  requiring  deriving  information  from 
the  data 

Respond  to  a  questionnaire  designed  to  obtain  the 
participants  perceptions  of  the  potential  relative  value  of 
Buckeye/LIDAR  and  CIB1/DTED2 

Weight  categories  and  criteria 

Participate  in  post-trial  debrief 


■^GEORGE 
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Measures  -  Objective 


•  Answers  to  questions  requiring  analysis  of  the  data  (H3) 

-  Imagery  Questions 

-  Elevation  Data  questions 

•  Need  for  additional  information  -  RFIs  (H2,  H4,  H5) 

-  Proxy  for  the  value  of  information  contained  in  the  data 

-  28  Criteria  in  6  categories 

•  Time  to  complete  scenario  (HI,  H4,  H5) 

-  Significant  in  prior  experiment 


•  Responses  to  a  questionnaire  evaluating  subjective  perception 
of  Buckeye/LIDAR  (H6) 


1 0  criteria 

Imagery  and  elevation 
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Accuracy  of  Information  (H3) 


•  In  all  cases  participants  were  able  to  derive  more 
accurate  information  from  Buckeye/LIDAR  data  than 
from  CIB1/DTED2  data  [p  <  0.001] 

-  Chi-Squared  tests  on  answers  to  questions 
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Percentage  of  Correct  Responses 

Buckeye 

LIDAR 

CIB1 

DTED2 

Overall 

72.80% 

15.60% 

Elevation 

74.40% 

23.40% 

Imagery 

71.20% 

7.80% 
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Requests  for  Additional  Information  (H2) 


•  Participants  using  Buckeye/LIDAR  required  less 
additional  information  [p  <  0.001],  on  average,  than 
when  using  CIB 1  /DTED2 

-  Buckeye/LIDAR  RFI  score:  4.26 

-  CIB1/DTED2  RFI  Score:  2.97 

•  RFIs  are  an  inverse  proxy  for  the  value  of  the 
information  contained  in  the  data. 

•  As  RFI’s  are  costly  in  time  and  manpower,  fewer 
RFIs  result  in  increased  tactical  flexibility,  improved 
force  security,  and  lower  demands  on  intelligence 
staffs 
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Time  to  Solution  (HI) 


Average  time  to  scenario  completion  (HI) 


-  Repeated  measures  ANOVA  [p  <  0.001] 

-  Buckeye/LIDAR:  51.67  min 

Time  to  Completion 

-  CIB1/DTED2:  47.40  min 


Average  difference  was  only 
4  min 

-  Higher  resolution  data 
required  more  time  to 
analyze 

Learning  effect  (H5) 

-  Average  time  to  completion 
was  shorter  for  the  second 
system  the  participants  used 

[p  =  0.01] 
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Subjective  Perception  (H6) 


There  is  strong  statistical  evidence  [p  <  0.001]  that,  when 
using  Buckeye  imagery  and  LIDAR  elevation  data, 
participants  believe  : 

they  can  produce  the  required  output  more  quickly 
it  is  easier  to  conduct  military  evaluations 
the  information  is  more  useful 


Summary 


“This  is  a  game  changer” 


Higher  resolution  imagery  and  elevation  data 
provides  : 

-  More  information  to  the  military  decision-maker 

-  Information  that  is  more  valuable  to  the  decision-maker 

The  reduced  costs  due  to  fewer  RFIs  would  probably 
overshadow  the  slightly  longer  analysis  time  required 
when  using  higher  resolution  data 

Participants  believe  that  higher  resolution  data 
improves  the  process  of  making  military  evaluations 
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Questions? 


